Neural networks can be trained to perform various artificial intelligence tasks, such as image recognition, speech recognition, handwriting recognition, and so on. Neural networks are typically trained using a large set of known data. The computational and memory resources required to train deep neural networks to reasonable accuracy in a practical amount of time are significant, and typically surpass the capabilities of a single commodity server. Reducing the computational and cache resource requirements may result in improved performance and scalability.